This our own evaluation of individual forecasts submitted to the EpiExpert model. This model is submitted to the German and Polish Forecast Hub each week.

These evaluations are preliminary - this means we cannot rule out any mistakes and the plots and analyses are subject to change. The evaluations are also not authorised by the German Forecast Hub team.

If you have questions or want to give feedback, please create an issue on our github repository.

Available Forecasts

Forecast Visualisation

Germany

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Poland

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Ranking by pairwise comparison

           model compare_against ratio pval adj_pval theta_i theta_j

1: EpiNow2 sophiemeakin NA NA NA 1 1 2: EpiNow2 Joel NA NA NA 1 1 3: EpiNow2 seb NA NA NA 1 1 4: EpiNow2 nib NA NA NA 1 1 5: EpiNow2 seabbs NA NA NA 1 1 —
221: anonymous_Cat anonymous_Cat 1 1 1 1 1 222: Sqxleaxes Sqxleaxes 1 1 1 1 1 223: hababylon hababylon 1 1 1 1 1 224: anonymous_Okapi anonymous_Okapi 1 1 1 1 1 225: anonymous_Shrew anonymous_Shrew 1 1 1 1 1

Score Ranking

Only those forecasters are named who have submitted a forecast in the most recent evaluated week.

Overall

Overall separate

Germany

Poland

Score Overview

Overview of all the models and metrics

Overall

Overall separate

Germany

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Poland

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Overiew of Scores by Horizon

Overview of all models and metrics stratified by forecast horizons

horizon = 1

Overall

Overall separate

Germany

Poland

horizon = 2

Overall

Overall separate

Germany

Poland

horizon = 3

Overall

Overall separate

Germany

Poland

horizon = 4

Overall

Overall separate

Germany

Poland

Interval Score Components

Weighted interval score broken down into its components “overprediction penalty”, “underprediction penalty” and “sharpness”

Overall

Overall separate

Germany

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Poland

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07

Calibration

Interval Coverage

Overall

Overall separate

Plot of the percentage of true values captured by each prediction interval

Quantile Coverage

Overall

Overall separate

Plot of the percentage of true values below each predictive quantile level

Bias

Overall

Overall separate

Separate by date

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

2020-11-30

2020-12-07